Diagnostic test accuracy of ultrasound for orbital cellulitis: A systematic review

Background Periorbital and orbital cellulitis are inflammatory conditions of the eye that can be difficult to distinguish using clinical examination alone. Computer tomography (CT) scans are often used to differentiate these two infections and to evaluate for complications. Orbital ultrasound (US) could be used as a diagnostic tool to supplement or replace CT scans as the main diagnostic modality. No prior systematic review has evaluated the diagnostic test accuracy (DTA) of ultrasound compared to cross-sectional imaging. Objective To conduct a systematic review of studies evaluating the DTA of orbital ultrasound compared with cross-sectional imaging, to diagnose orbital cellulitis. Methods MEDLINE, EMBASE, CENTRAL, and Web of Science were searched from inception to August 10, 2022. All study types were included that enrolled patients of any age with suspected or diagnosed orbital cellulitis who underwent ultrasound and a diagnostic reference standard (i.e., CT or magnetic resonance imaging [MRI]). Two authors screened titles/abstracts for inclusion, extracted data, and assessed the risk of bias. Results Of the 3548 studies identified, 20 were included: 3 cohort studies and 17 case reports/series. None of the cohort studies directly compared the diagnostic accuracy of ultrasound with CT or MRI, and all had high risk of bias. Among the 46 participants, diagnostic findings were interpretable in 18 (39%) cases which reported 100% accuracy. We were unable to calculate sensitivity and specificity due to limited data. In the descriptive analysis of the case reports, ultrasound was able to diagnose orbital cellulitis in most (n = 21/23) cases. Conclusion Few studies have evaluated the diagnostic accuracy of orbital ultrasound for orbital cellulitis. The limited evidence based on low quality studies suggests that ultrasound may provide helpful diagnostic information to differentiate orbital inflammation. Future research should focus studies to determine the accuracy of orbital US and potentially reduce unnecessary exposure to radiation.


Introduction
Periorbital and orbital cellulitis are inflammatory conditions of the eye that can be difficult to distinguish using clinical examination alone, particularly in young children [1]. Clinically, both conditions present with redness and swelling of the eye, but their etiology, management and prognosis are different. In cases of diagnostic uncertainty or to evaluate for orbital cellulitis associated complications (e.g., subperiosteal abscess), cross-sectional imaging is often obtained. Historically, a Computed Tomographic (CT) scan with contrast has been the modality of choice as it is readily available, does not require sedation in young children, provides excellent anatomical differentiation between bones and soft tissue, and can identify inflammation [2]. However, CT scans expose patients to ionizing radiation, which is a concern in pediatric patients due to the association with malignancy [3]. More recently, magnetic resonance imaging (MRI) use has increased, although it appears to be used in addition to CT scans in cases which require repeat diagnostic imaging [4]. In practice, the use of MRI is limited by its availability, cost, repeat imaging after CT [5,6] and the need for general anesthesia in young children [7].
Ultrasound may offer an alternative imaging modality with the potential to quickly and easily diagnose patients with periorbital versus orbital cellulitis without radiation exposure or need for general anesthesia. Several early studies advocated in favor of ultrasound as a diagnostic tool for ocular inflammation, highlighting its portability and ability to reduce time from presentation to initiation of treatment [8][9][10]. Despite these promising earlier studies, ultrasound has not become common practice for diagnosis and treatment. Ultrasound as an imaging technology has improved substantially over the last three decades, with recent advances in portability, cost, and access. Physicians are increasingly using bedside ultrasound or point-of-care ultrasound (POCUS) across multiple specialties, including emergency medicine, general practice, pediatrics, and internal medicine [11]. Recent diagnostic test accuracy studies of POCUS for ocular assessments have identified high sensitivity and specificity for certain conditions in adults, such as retinal detachment and vitreous hemorrhage [12,13].
However, there is limited contemporary research on the diagnostic test accuracy of orbital ultrasound for orbital cellulitis, a clinical area that can be a diagnostic challenge. Given the increasing access and use of ultrasound, it may provide an adjunctive diagnostic tool for clinicians to guide diagnosis and management and could be used for serial assessment to evaluate response. Therefore, our research question/aim was to determine the diagnostic test accuracy of orbital ultrasound in the diagnosis of orbital cellulitis and its complications (e.g., subperiosteal abscess) compared to a diagnostic reference standard (i.e., computed tomography [CT] or magnetic resonance imaging [MRI]) in patients presenting to the emergency department or admitted to hospital with a suspected severe orbital infection.

Search methodology
We performed a systematic review in accordance with the preferred reporting for systematic reviews and meta-analysis (PRISMA-DTA) guidelines. The PRISMA-DTA checklist has been provided as a S1 Appendix) [14].

Eligibility criteria, data extraction and risk of bias assessment
Patients of any age who were seen at an emergency department (ED) or hospital (any level of care) with suspected orbital infection (i.e., periorbital or orbital cellulitis) were eligible for inclusion. Studies were included if the study participants had undergone an ultrasound (index test) and a diagnostic reference standard (i.e., CT or MRI) within a reasonably close time interval (i.e., defined as within 48 hours). We included any type of ultrasound including those obtained by technologists and point-of-care ultrasounds (POCUS) obtained by healthcare professionals. We included all study types (i.e., case reports, case series, observational studies, randomized controlled trials. There were no restrictions based on country of origin or language of publication. We used Covidence to manage the selected studies [15]. Two review authors (MRA and TAO) independently screened titles and abstracts. The full text of any potentially relevant articles that met the inclusion criteria were then reviewed for eligibility. The review authors (MRA and TAO) independently screened the articles for inclusion and resolved any discrepancies through discussion. When needed, a third review author (PJG) was consulted. The review authors (MRA and TAO) independently extracted data using a custom designed data collection tool (S3 Appendix). Key aspects included in the data extraction form included general characteristics (e.g., year, country of study, time period, study design), population (e.g., age, sex, inclusion/exclusion criteria), imaging modality findings (e.g., signs of inflammation, abscess), timing of diagnostic tests, types of ultrasound probe/ transducer, and outcomes (e.g., definitions of orbital cellulitis and complications). The imaging findings indicated the clinical diagnosis as per Chandler's criteria [16]: I) periorbital (preseptal) cellulitis; II) orbital cellulitis; III) sub-periosteal abscess; IV) orbital abscess; and V) cavernous sinus thrombosis. These distinctions are anatomically based but provide a helpful schematic upon which to evaluate the diagnostic role of ultrasound. We also extracted data on ultrasound techniques that were employed to better visualize the orbit. Potentially relevant studies that were identified but for which the full text could not be obtained despite multiple attempts to contact the authors, were first labelled as "awaiting classification" and subsequently excluded.
Two reviewers (MRA, PJG) evaluated the methodological quality of individual studies using the Quality Assessment of Diagnostic Accuracy Studies (QUADAS-2) tool [17] and The Joanna Briggs Institute Critical Appraisal tools [18]. Discrepancies were resolved through discussion.

Statistical analysis
For the primary analysis, we a priori planned to use the individual patient as unit of analysis. For the secondary analysis, we used individual eyes as the unit of analysis. We planned to use Review Manager Web version (RevMan Web) to plot estimates of sensitivity and specificity from the studies in receiver operating characteristics (ROC) space and to build forest plots [19]. As the results of the ultrasound scan were expected to be binary, (i.e., presence or absence of orbital cellulitis), we intended to perform meta-analysis using a bivariate model to estimate summary sensitivity (i.e., proportion of individuals with orbital cellulitis who are correctly detected by ultrasound) and specificity (i.e., proportion of individuals without orbital cellulites who are correctly identified by ultrasound) [20]. However, due to insufficient data, we were unable to conduct a meta-analysis. We planned to conduct a meta-analysis if there were sufficient number of studies with low clinical heterogeneity. However, due to limited number of included studies, the results section is primarily descriptive.
Planned sensitivity analyses included the effect of excluding studies where older generations of ultrasound probes were used, and the effect of excluding studies with a high risk of bias for primary outcome. Planned subgroup analyses included the following characteristics: 1) Study type: experimental vs observational; 2) Age: <18 yrs (pediatric) vs. >18 yrs (adult); and 3) Type of ultrasound: standard ultrasound vs. POCUS.

Results
The search yielded 3548 studies, of which 215 underwent full-text screening, and 20 were included. Among these 20 articles, 3 were cohort studies and 17 were case reports/series. The PRISMA diagram in Fig 1 illustrates the process for study selection. Characteristics of the included cohort studies are outlined in S4 Appendix.

Cohort studies
All three identified retrospective cohort studies were published before 2002, and none reported the sensitivity and specificity of ultrasound. Among the 46 patients reported in these studies, both ultrasound and CT scans were conducted on 18 (39%) cases. A summary of the cohort studies is provided in Table 1. Goodwin et al. was published in 1982 and reviewed medical records of 22 patients diagnosed with orbital cellulitis. Age of the patients ranged from 5 to 68 years. Orbital ultrasound was conducted by experienced technicians and CT scans were reviewed by a neuroradiologist. Findings were in agreement in five of the seven cases where both ultrasound and CT scans were conducted. In the remainder, ultrasound identified wooden foreign body and early abscess formation that were missed in the CT scans. Kaplan et. al. conducted a similar study published in 1999. Seven children between 2 and 13 years old hospitalized with sinus induced orbital infection were examined regularly by an otolaryngologist and ophthalmologist. All children underwent contrast enhanced CT scan and orbital ultrasound by an ophthalmologist within 24 and 48 hours of admission, respectively. Findings from the ultrasound and CT scan were in agreement in three. In two patients with orbital cellulitis (Chandler II) and two patients with Subperiosteal abscess (Chandler III), the authors reported that CT scan could not differentiate between orbital cellulitis and subperiosteal abscess, whereas ultrasound was able to do so. Mair et. al., published in 2002, reviewed clinical records of 17 children aged 1 to 10 years presenting with swelling and erythema of the eyelids. Orbital ultrasound and additional imaging (CT or MRI scan) were performed in only four patients. Although comparison of imaging findings for these patients were not reported, the authors reported that ultrasound identified patients with orbital infection. The authors also suggested that they were able to differentiate between subperiosteal abscess (Chandler III) and subperiosteal inflammatory infiltration (Chandler II) based on the sonographic findings.
QUADAS-2 assessment of the three cohort studies is summarized in Fig 2. For the patient selection domain, there was unclear risk of bias in two studies given the lack of detail on the process of patient selection. For the index test domain, all three studies had an unclear risk of bias as it was unclear from the study methods whether ultrasound (index test) was conducted before or after CT or MRI (reference standard). Risk of bias was low for the reference standard domain in two studies. The studies did not clearly state if the reference interpreters were aware of the index test. Risk of bias for 'flow and timing' was judged to be high in one study and unclear in the other two. We did not notice any major applicability concerns in any of the studies.

Case reports/series
There were 17 case reports/series including a total of 23 patients published between 1982 and 2021. Most reports were from hospitals in the USA (n = 8) and included patients ranging from birth to 83 years. A total of 11 males and 11 females were reported in the studies; the sex was not reported in one study. Although a direct comparison between ultrasound and other imaging modalities was not the objective of the studies, the authors report that most ultrasound findings were in agreement with cross-sectional imaging. The Chandler classification was not always specified; however, the authors did acknowledge that the ultrasound findings were sufficient to reach an immediate diagnosis. We have summarized the descriptive findings of case reports in Table 2.
The critical appraisal of case reports revealed that most of the cases were well reported. The authors defined the patient demographics and described the diagnostic tests clearly. Only three of thirteen studies did not clearly describe the outcome of the patient post intervention. Critical appraisal of the four case-series identified major concerns. It was either unclear or the authors did not report if consecutive patients were included and case inclusion were unclear. However, patient demographics, measurement of the clinical condition, and follow up were clearly reported. The appraisal of the case reports and case-series are presented in Tables 3  and 4.

Guidance on ultrasound technique
Some of the studies reported the brand, model, and frequency of the transducer used to examine the patients, but it was difficult to compare or reach conclusions due to substantial variation across the ultrasound devices. Earlier studies do however suggest that a high-frequency (>10 Mhz) linear array transducer could be used to examine the orbital region [9,10]. While the studies provided limited information on diagnostic performance, the authors report that most of the patients were examined in supine position without sedation. Some techniques were suggested that could guide better orbital assessment with ultrasound. For instance, the authors suggested to place the transducer on the closed upper eyelid using a non-irritating gel  (6) Routine showed increased orbital fat density and volume around the superotemporal orbital rim. There was also a suggestion of hypertrophy of the superior and lateral recta. There was no evidence of thickened sclera and sub-tenon's space was normal.

CT (-)
The muscle bellies as well as the tendons of the medial and inferior recta were diffusely thickened. This suggested orbital myositis rather than thyroid eye disease. and to avoid any pressure on the eyeball and orbital tissues [10,21,22]. A transparent adhesive dressing (Tegaderm) could also be gently applied over the closed eye before applying the gel, ensuring no air is trapped between the plastic and the skin [22]. The medial orbital wall can be visualized by moving the transducer as far as possible toward the lateral canthus of the eye [10]. Another author pointed out that, while imaging the eye in the standard plane using POCUS, no obvious anomaly was detected. However, by rocking the probe medially and laterally and focusing on the nasal and temporal tissues additional views could be obtained. This alternating motion revealed prominence and edema of the nasal post septal tissue with echogenic, inflammatory fat in an adult female patient with orbital cellulitis [23]. For the transverse and longitudinal planes, the marker of the transducer can be oriented towards the right or head of the patient, respectively. The transducer can also be fanned back and forth for clear visibility of anterior and posterior anatomy of the eye. For assessing extraocular muscles, the patient can be asked to move their eye or fix their gaze on objects. The same study suggested that ocular POCUS could be performed while distracting the child by a video on the parent's smartphone [22]. Studies have also suggested against use of color flow or doppler mode to limit the theoretical risk of damaging tissues [24,25]. A table listing all the ultrasound techniques have been provided in the S5 Appendix.

Discussion
This systematic review of the diagnostic test accuracy of orbital ultrasound to diagnose orbital cellulitis found limited, low-quality evidence of comparable performance to conventional cross-sectional imaging. Only 3 cohort and 17 case reports/series were identified; there were no prospective observational studies or randomized controlled trials. Furthermore, included studies were several decades old and used orbital ultrasound mainly as a supplementary imaging modality. Despite the limitations, the studies highlight the potential value of orbital ultrasound in both diagnosis and management of severe orbital infections [8,10].
Several authors noted the benefit in choosing ultrasound as it is portable and accessible and does not require sedation in pediatric patients [10]. The diagnostic workup is rapid which can shorten the interval between clinical presentation and initiation of treatment, especially in patients with non-specific clinical findings. Orbital cellulitis is considered an ophthalmologic emergency which requires urgent antibiotic therapy, which if left untreated, or if treatment is delayed, can be both sight-and life-threatening. In this clinical scenario, a cost-effective, rapid, low risk, bedside procedure such as ultrasound might be useful. Ultrasound may be especially useful in young children and uncooperative patients who would require sedation or a general anesthetic [22]. Another benefit of ultrasound is its use for reassessment and evaluation of response to treatment [8,21,26]. Compared with CT scan, ultrasound provides useful information but without radiation exposure and with less cost [8]. Furthermore, if ultrasound can differentiate pre and post septal infections, this may eliminate the need for additional crosssectional imaging with CT or MRI.
POCUS can be performed at the bedside, does not require sedation, and is especially useful in children [27]. Pediatric emergency medicine has recognized the potential of POCUS to improve care for their patients [28]. POCUS is also gaining popularity in the inpatient setting. Its benefits in clinical decision-making and patient management are increasingly recognized [29]. For example, medical schools have also started incorporating POCUS into their undergraduate medical education [30]. POCUS is now being used to detect cranial hemorrhage in preterm infants [27] and skull fractures in toddlers [31]. POCUS has also been found to be beneficial in investigating orbital trauma as it can identify orbital fractures and correlates well with CT [32]. POCUS demonstrates a high sensitivity and specificity in the diagnosis of traumatic eye injury, reporting an 85% sensitivity, 98% specificity, and 97% accuracy in detection of traumatic lens dislocation [33]. Optic nerve sheath diameter may be measured accurately with transorbital ultrasound for the detection of raised intra-cranial pressure or diagnosis of papilledema [34][35][36][37]. POCUS is also used to facilitate interventions safely, such as obtaining vascular access and endotracheal intubation in children [27]. Ultrasound is not limited to making initial diagnoses, but is also used to evaluate response to treatments [38,39].
Like any diagnostic test, orbital ultrasound has limitations. While it may differentiate between pre-and post-septal infection, it cannot sufficiently assess the orbital apex and the paranasal sinuses. It also cannot define the abscesses of the posterior orbit, nor can it assess the intracranial extension of orbital infection, which is an important case of morbidity [9]. Patients who fail to improve with appropriate care, those with neurologic symptoms, or showing signs of clinical deterioration, should still undergo a CT or MRI scan as part of complete diagnostic workup.

Limitations
Our systematic review has several limitations. First, we had to exclude 37 studies that were awaiting classification as we could not obtain the full-text despite multiple attempts at contacting study authors and reference libraries. Most of these studies were completed several decades ago. Second, the exact timing of the reference standard test was often unclear given that most studies were not designed to compare ultrasound to other imaging modalities. Third, several included studies were decades old, and the imaging technologies might not be representative of those currently used. Fourth, many of the included studies were of low quality with a high risk of bias; no prospective, blinded, diagnostic accuracy studies were identified. However, such studies are feasible given similar studies in orbital ultrasound for retinal detachment in adult care [12,40]. Studies on the application of ultrasound are susceptible to publication bias, as studies that report utility of a diagnostic tool are more likely to be published compared with studies that report it is not useful.
While the evidence generated from this review is limited, it is the first to systematically assess the benefit of ultrasound compared other imaging modalities to diagnose orbital cellulitis. Our findings support the need for future prospective observational studies or trials to evaluate the diagnostic test accuracy of ultrasound to diagnose and evaluate response to treatments in orbital cellulitis.